Confidence Estimation in Classification Decision: A Method for Detecting Unseen Patterns

The classification task for a real world application shall include a confidence estimation to handle unseen patterns i.e., patterns which were not considered during the learning stage of a classifier. This is important especially for safety critical applications where the goal is to assign these situations as ”unknown” before they can lead to a false classification. Several methods were proposed in the past which were based on choosing a threshold on the estimated class membership probability. In this paper we extend the use of Gaussian mixture model (GMM)to estimate the uncertainty of the estimated class membership probability in terms of confidence interval around the estimated class membership probability. This uncertainty measure takes into account the number of training patterns available in the local neighborhood of a test pattern. Accordingly, the lower bound of the confidence interval or the number of training samples around a test pattern, can be used to detect the unseen patterns. Experimental results on a real-world application are discussed.